Method for producing video coding and programme-product

ABSTRACT

According to the invention, there are provided sets of contexts specifically adapted to encode special coefficients of a prediction error matrix, on the basis of previously encoded values of level k. Furthermore, the number of values of levels other than 0 is explicitly encoded and numbers of appropriate contexts are selected on the basis of the number of spectral coefficients other than 0.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/786,376 filed on Feb. 10, 2020, which is a divisional of U.S. patentapplication Ser. No. 16/181,392 filed on Nov. 6, 2018, which is adivisional of U.S. patent application Ser. No. 15/859,349 filed on Dec.30, 2017, which is a continuation of U.S. patent application Ser. No.15/148,606 filed on May 6, 2016, which is a continuation of U.S. patentapplication Ser. No. 10/489,651 filed on Mar. 15, 2004, which is a U.S.National Stage Application of International Application No.PCT/DE02/03385 filed Sep. 11, 2002, which designates the United Statesof America, and claims priority to DE Application No. 102 18 541.7 filedApr. 25, 2002 and DE Application No. 101 45 373.6 filed Sep. 14, 2001,the contents of which are hereby incorporated by reference in theirentirety.

BACKGROUND OF THE INVENTION

Predictive video encoding procedures use areas of an image alreadytransmitted to estimate the current image information and calculate aprediction error which deviates therefrom. As a rule, the current imageinformation is determined from the image areas already transmitted bydividing the current input image into blocks and by looking for blocksin previous images that correspond to each of these blocks, whichminimizes the extent of an error or a gap. The block image constructedin this way is subtracted from the current input image and theprediction error thus calculated is transformed via a discrete cosinetransformation or an integer transformation from the local into thefrequency range. The transformed prediction error data is then quantizedand the coefficients thus produced are compressed and sent to therecipient by a Context Adaptive Binary Arithmetic Coder (CABAC).

For arithmetic encoding, the coefficients contained in the predictionerror matrix are linearized by coefficient sampling and converted into aseries of levels and lengths of zero sequences. Both the level valuesand the length values are represented here as unary values and encodedindependently of each other bit-by-bit. The leading signs of the levelsare encoded separately. For encoding the level values, a specificcontext is used in each case for the first two bits and a furtherspecific context is used for all following bits. Context in thisconnection should be understood as the distribution of the frequenciesof the logical 0s and 1s. The context thus specifies the likelihood thata bit is set or not set. For encoding the length values, a specificcontext is used for the first bit and a further specific context is usedfor all following bits. A separate context is also used for encoding theleading sign which is represented by a individual bit. The six specificcontexts used together make up a context set.

Using this prior art as a starting point, an object of the presentinvention is to further improve context-adaptive binary arithmeticencoding.

SUMMARY OF THE INVENTION

In accordance with the present invention, such object is achieved by amethod with the following steps:

-   -   providing of a prediction error matrix;    -   converting of the prediction error matrix by coefficient        sampling into a series of symbols; and    -   performing context-adaptive arithmetic encoding of the symbols        on the basis of symbol frequencies, for which the distribution        is selected depending on an already encoded symbol.

The present invention is based on the knowledge that there arestatistical dependencies between consecutive symbols since largecoefficient values occur predominantly at the start of coefficientsampling. The statistical dependencies between the symbols can beexploited by, depending on the symbols transmitted beforehand, usingspecific distributions of the symbol frequencies as a basis forcontext-adaptive arithmetic encoding. In contrast to the prior art, thedistributions of the symbol frequencies used for encoding are not solelyselected depending on the position of the symbol to be encoded withinthe symbol sequence, but also depending on a symbol actually transferredpreviously in each case.

In a preferred embodiment, the prediction error matrix is converted bycoefficient sampling into a series of levels and lengths and the levelvalues encoded depending on the value of a previously transmitted leveland the length values depending on the value of a previously encodedlength.

It should be pointed out that level value in this context should betaken as the amount of a level. Since the lengths cannot assume negativevalues, the length values are occasionally referred to below as lengthsfor short.

Since the statistical dependencies between the level values and lengthvalues are particularly prominent, this procedure makes particularlyefficient encoding possible.

In a further preferred embodiment, the levels are sorted according tosize and the statistical dependencies between the level values thusstrengthened.

Finally, there is provision for determining and encoding the number ofcoefficients. This procedure also allows the selection of the frequencydistribution for encoding the symbols to be made depending on the numberof coefficients.

Additional features and advantages of the present invention aredescribed in, and will be apparent from, the following DetailedDescription of the Invention and the Figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a block diagram of an encoder.

FIG. 2 shows a block diagram which illustrates the encoding ofcoefficients of a prediction error matrix.

FIG. 3 shows a diagram exemplifying the sampling process of theprediction error matrix.

FIG. 4 shows a global frequency distribution of the level values.

FIG. 5 shows a frequency distribution of the level values depending onthe previous level value.

FIG. 6 shows an illustration of a context set used for encoding thelevel values.

FIG. 7 shows an illustration of a context set used for encoding thelengths.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 shows an encoder 1 which operates in accordance with theprinciple of the movement-compensating hybrid encoding. The encoder 1has an input 2 via which the video data is fed to encoder 1. A movementestimation unit 3 downstream from input 2 segments a current image ofthe video data stream to be encoded into rectangular macro blocks. Foreach of these macro blocks, the movement estimation unit 3 looks formatching macro blocks from the images already transmitted and calculatestheir movement vectors. The movement vectors can be encoded with the aidof an encoding unit 4 and embedded via a multiplexer 5 into a bit streamoutput at an output 6. The movement vectors of the macro blockscalculated by the movement estimation unit 3 are also notified to amovement compensator 7 which, starting from the images alreadytransmitted stored in an image memory, 8 calculates the prediction imageproduced by the movement of the macro blocks of the images alreadytransmitted. This prediction image is subtracted in a subtractor 9 fromthe original image in order to create a prediction error which is fed toa discrete cosine transformer 10 with downstream quantizer 11. Theforecasting error is also referred to as the prediction error ortexture. The transformed and quantized prediction error is forwarded toa further context-sensitive encoding unit 12 which converts theprediction error matrix with the aid of context-adaptive binaryarithmetic encoding into a bit stream which is fed into the multiplexer5 and is embedded into the bit stream output at the output 6.

Processing in the discrete cosine transformer 10 converts the macroblocks with, for example, 8×8 pixels into a prediction error matrix with8×8 spectral coefficients. In this case, the first coefficient is giventhe average brightness of the macro block. The first coefficient is alsoreferred to as the direct component. The remaining spectral coefficientsreflect higher frequency components of the brightness distribution withincreasing index number, which is why they are referred to asalternating components.

The data rate is further reduced by the subsequent quantizer 11. Withplanar image elements; the prediction error only changes slowly frompixel to pixel so that, after processing in quantizer 11, most of thehigh-frequency spectral coefficients are equal to zero and thus do noteven have to be transmitted.

The quantizer 11 additionally takes account of psychovisual effects. Thehuman brain perceives low-frequency image components (namely, extendedareas of image components) far more clearly than high-frequency imagecomponents; in particular, details. Thus, the high-frequency spectralcoefficients will be quantized more roughly than the low-frequencyspectral coefficients.

To correct the images stored in the image memory 8, the spectralcoefficients will be fed to an inverse quantizer 13 and an inversediscrete cosine transformer 14 and the data reconstructed from theprediction error matrix in this way added in an adder 15 to theprediction image created by the movement compensator 7. The image thuscreated corresponds to the image produced on decoding. This image isstored in the image memory 8 and is used by the movement estimation unit3 as a basis for calculating the movement vectors of the followingimages.

The function of the context-sensitive encoding unit 12 is describedbelow on the basis of FIGS. 2 through 7.

The context-sensitive encoding unit shown in FIG. 2 has a samplingdevice via which the spectral coefficients in the transformed predictionerror matrix are converted into a series of levels and lengths of zeros.This type of diagram is also referred to as a run/level diagram. Thesampling device 16, for example, converts the series of spectralcoefficients 20-1001000000 into the series (0/2) (1/−1) (2/1) (0/0). Inthis case, the number before the forward slash specifies the number ofzeros before the level specified after the forward slash. The numbersbefore the forward slash are referred to as lengths. The 0 specified inthe last number pair after the forward slash identifies the situationwhere the rest of the sequence of digits consists purely of zeros. Thelast pair of digits can be viewed as an entry identifying the end of theblock transmission (EOB=End of Block).

The sequence of levels and lengths created by the sampling unit 16 isfed to a converter 17 which converts the binary representation into aunary (single value) representation. In this case, the pairs of numberslisted in the example are encoded, in each case, in this sequence level,length of sequence of zeros and leading sign of the level. The pair ofnumbers (0/2) will then be converted in this case into the unarysequence 110/0/0 and the pair of numbers (1/1) into the sequence10/10/1.

The unary sequence of digits is finally fed to an arithmetic encoder 18which performs the actual context-adaptive arithmetic encoding. Toperform the context-adaptive arithmetic encoding, the arithmetic encoder18 needs the frequency with which the ones and zeros have occurred orwithin the unary data stream delivered by a converter 17 in each case.This probability and thereby the distribution of the frequencies of theones and zeros is delivered to the arithmetic encoder 18 by the analyzer19 which is accessed by the sampling device 16 with the series of levelsand the lengths and, from this, determines current distributions of thelikelihood for the occurrence of logical zeros and ones.

In the encoding unit 12 described here; statistical dependencies betweenthe levels and lengths will be taken into consideration in a particularway. These statistical dependencies are based on the fact that largelevel values occur mostly at the start of the sampling process.Conversely, large length values are more frequent at the end of thesampling process. Since, in accordance with FIG. 3, the transformedprediction error matrix 20 will be sampled with a zig-zag pattern 21through which initially the low-frequency spectral coefficients and thenthe higher frequency spectral coefficients will be read out, Large levelvalues then occur above all at the beginning of the sampling process;that is, at the beginning of the series of levels and lengths.

FIG. 4 shows a diagram in which a distribution curve 22 specifies thefrequency distribution P(k) depending on the level values k. Itgenerally applies that the likelihood of small level values is greaterthan that of large level values. The distribution curve 22 thus dropsmonotonously starting from a maximum value at the level value k=0.

FIG. 5 contains a further diagram showing a distribution curve 23 whichspecifies the frequency distribution P(klk=kX) for the likelihood thatthe level value of k will occur after a level value of k=kX. Thisdistribution curve 23 features a maximum that lies at a value k<kX. Assuch, after the occurrence of a level value k=kX, lower level values arevery likely. This corresponds to the fact that the level values droptowards the end of the sampling process.

Because of the statistical interdependency of the level values, it makessense to also select the symbol frequencies for the zeros and ones onwhich the context-adaptive arithmetic encoding in the arithmetic encoder18 is based depending on the value of the previously encoded level.

The statistical dependencies can be strengthened even more when thelevel values are sorted according to size. Suitable sorting proceduresin which the level values are sorted according to size, and in which thesorting information is transmitted separately, are known to the expertsand are not as such part of the present invention. In addition, futuresorting processes for the application are also considered in thiscontext.

By sorting the levels, the part of the curve 24 shown in FIG. 5 as across-hatched area is truncated to an extent. This strengthens thestatistical dependencies of the level values further.

The statistical dependencies do not just relate to the level values butalso to the lengths. As already mentioned, large length valuesparticularly occur toward the end of the sampling process. As such, italso makes sense to select the distribution of the frequencies for thezeros and ones on which the arithmetic encoding in the arithmeticencoder 18 is based depending on the value of the previously encodedlengths.

FIGS. 6 and 7 show diagrams of the context sets which are used forencoding the level values and length values.

In FIG. 6, the context sets on which the coding of the sorted levelvalues is based are shown. In the case, n=2 is selected for the numberof individually encoded bit positions and m=4 is selected for the numberof individually encoded level values. As already mentioned, the encodingof a level value 1 is undertaken depending on the amount of thepreviously encoded level k.

It should be noted that with the unary representation of sorted levelvalues, the concluding zero used for the unary representation of 1 canbe omitted if 1=k applies, since the maximum value for 1 is equal to k.

For the first min(n, k) bits, a separate context is used in each case.For all possible following bits up to the maximum length of the unaryrepresentation of 1, a collective context will be used. In FIG. 5 thisis the collective context for the third and the following bits. Trialshave shown 2 to be a good value for n. The leading sign is representedby an individual bit and a specific leading sign context is used forencoding the leading sign. The context set for encoding a level with anamount 1, provided the last encoded level had the amount k, thusincludes min(k+1, min(n,k)+2) context and is referred to below asLevelContextSet(k).

For the encoding of the first level read out of the transformedprediction error matrix it is not possible to refer back to a previouslyencoded level so that the first level value read out must be encodedseparately. As a context for the first level value to be encoded, thecontext of the largest possible level value is assumed, which to someextent is possible by the transformation and the subsequent quantizing.

Otherwise, instead of the different contexts for a k>m, the context fork=m can be used as a collective context. For encoding a level value 1depending on the previous level value k, the following context set inaccordance with FIG. 6 is thus used. LevelContextSet(min(m,k)). Trialshave shown m=4 to be a good value.

Basically, however, there is also the option of a dynamically designingthe parameters n and m and transmitting the values selected for them ina header.

The encoding of a length r is undertaken depending on the previouslyencoded length p. The following applies for the first length: p=0. Asfor the level values, a separate context is also used for the first nbits in each case. For all following bits a common context is used. n=3has proven itself in trials. The context set for encoding a length runder the condition that the last encoded length has the amount p, isdesignated below by RunContextSet(p). For encoding a length r dependingon p, the following context set will be used: RunContextSet(min(m,p)).In trials m=5 has proved to be a good value.

Basically, there is also the option of dynamically designing theparameter m in this connection and transmitting the values selected forit with the aid of a suitable information element.

Through the method described here in which the contexts used forencoding are selected depending on the previously encoded level value orlength value, the data rate can be reduced by 2 to 3%.

A further reduction can be achieved when the end of this sequence oflevels and lengths read out from the prediction error matrix is notencoded using the level value 0, but when the sequence is preceded bythe number of read out levels different from 0. In the example givenabove, this would then not produce the sequence (0/2)(1/−1)(2/1) (0/0),but 3(0,2)(1/−1)(2/1).

In this case, the information about the number of level values differentfrom 0 can be used for an efficient encoding of the level values.Because transformed prediction error matrices with a few spectralcoefficients differing from 0 have as a rule only spectral coefficientswith very low level values. It thus makes sense, depending on the numberof spectral coefficients differing from 0, to now switch between thedifferent context sets and thus improve the efficiency of the encodingprocess.

Alternatively, it is also possible, depending on the amount of the firstlevel values transmitted, to switch between different context sets.

Taking into account the number of coefficients differing from 0 in theencoding by correspondingly adapted context sets, reduces the data rateof the bit stream output at output 6 by a further 5 to 6%.

The devices and methods described here are suitable for use within theframework of existing video standards such as H.263 and MPEG-4, as wellas H.26L. The methods and devices described here are, however, alsosuitable for use in future standards corresponding to the currentstandards.

The method described here and the devices described here are especiallyefficient at low data rates since very many levels at low data rateshave a value of 1 for the amount. This has not, however, been able to bestatistically modelled efficiently thus far since relating the lastencoded level value to the range of values of the currently to beencoded level values is not possible without prior knowledge about thesequence of the coefficients.

Finally it should be pointed out that the devices and methods describedhere can be achieved both in hardware and in software form.

Although the present invention has been described with references tospecific embodiments, those of skill in the art will recognize thatchanges may be made thereto without departing from the spirit and scopeof the present invention as set forth in the hereafter appended claims.

What is claimed is:
 1. A device for video coding for a current imagerepresented by macroblocks and part of a data stream, the devicecomprising: a processor; and a machine-readable memory, the memorystoring instructions that, when loaded and executed by the processor,cause the processor to: receive motion vectors for the macroblock of thecurrent image based on one or more previously received reference images,wherein at least one previously received reference image is stored in animage memory; calculate a prediction image based on the received motionvectors, the prediction image representing predicted movement of themacroblocks of the reference image based on the received motion vectors;provide a prediction error matrix, indicating a prediction error aftertransformation by a transformer and quantization by a quantizer, theprediction error representing a subtraction result by subtracting theprediction image from the current image by a subtractor; convert theprediction error matrix by coefficient sampling into a series ofsymbols; perform context-adaptive arithmetic encoding of the symbols onthe basis of symbol frequencies with an already-encoded symbol, thealready-encoded symbol appearing immediately before a given symbol forwhich context-adaptive arithmetic encoding is performed, and form a bitstream by the context-adaptive arithmetic encoding of the symbols.
 2. Adevice in accordance with claim 1, wherein the prediction error matrixis converted by coefficient sampling into a series of levels and lengthsof zero sequences.
 3. A device in accordance with claim 2, wherein thelevels and lengths are represented as unary values.
 4. A device inaccordance with claim 2, wherein each of the levels is context-adaptedarithmetically encoded, wherein a distribution of underlying level valuefrequencies depends at least in part on a level value previouslyencoded.
 5. A device in accordance with claim 2, wherein each of thelengths is context-adapted arithmetically encoded, wherein adistribution of underlying length value frequencies depends at least inpart on a length value previously encoded.
 6. A device in accordancewith claim 2, wherein the level values are sorted before thecontext-adaptive arithmetic encoding using 2-level values.
 7. A devicein accordance with claim 2, wherein a count of the symbols read out forcoefficient sampling is encoded and transmitted.
 8. A device inaccordance with claim 7, wherein selection of the distribution of symbolfrequencies depends at least in part on the amount of the symbols readout.